Systems and methods to enhance segment during trick play

ABSTRACT

Systems and methods are provided herein for including supplemental content with segments based on the complexity of the segment. This may be accomplished by a system receiving complexity information related to a media asset and user information related to a viewer to determine if one or more segments of the media asset is complex for the user. If the system receives a trick play command, from the user, during a segment categorized as complex, the system can use the complexity information and user information to generate supplemental content, facilitating better user understanding of the complex segment.

BACKGROUND

The present disclosure relates to trick play features such as rewind,pause, and fast-forward, and in particular to techniques for providingsupplemental content during trick play.

SUMMARY

Whether it is to allow a user to pause a movie to go make popcorn orrewind a television show to re-watch an action scene, trick playfeatures (e.g., pause, rewind, fast-forward, etc.) have becomecommonplace for viewers consuming media. Many users utilize trick playfeatures to help them better understand the content they are viewing.Trick play features are especially helpful when a user is viewing amedia asset with complex topics, ambiguous and/or unclear cinematography(e.g., the scene progressing too quickly, the focal point of the scenebeing unstable, etc.), and/or other complex elements. For example, if auser is watching a movie (e.g., “The Matrix”), which is related tocomplex topics the user is unfamiliar with (e.g., future technology,artificial intelligence, etc.), the user can rewind complex scenes inthe hopes of better understanding the complex scenes. However, there isno guarantee that rewinding complex scenes, or pausing a complex sceneto search the Internet for explanations, will help the user betterunderstand the complex scene. This sub-optimal user experience can leadto frustrated users giving up on complex media. In view of thesedeficiencies, there exists a need for improved systems and methods forincluding supplemental content with complex media segments during trickplay to facilitate better user understanding of the complex segments.

Accordingly, techniques are disclosed herein for including supplementalcontent with segments during trick play based on the complexity of thesegment. To facilitate the inclusion of supplemental content withcomplex segments during trick play of a media asset, the system can useone or more complexity factors to determine if a given segment in themedia asset is complex. These complexity factors may include the viewinghabits of users who have watched the media asset, reviews related to themedia asset, metadata associated with the media asset, or similar suchcomplexity-related information. When the media asset is made availableto the public, the system can monitor (e.g., via Live Logging data)viewing habits of users for trick play trends. For example, if a numberof users rewind a first segment of the media asset, the system candetermine that the first segment is a complex segment. When determiningthe complexity of a segment, the system may also consider whether thereare reviews available for the media asset. For example, online platforms(e.g., Twitter, Facebook, IMDb, Netflix, Prime, TiVo, etc.) often havereviews where they explain and/or discuss complex segments of mediaassets. These reviews can be used to determine if certain segments ofthe media asset are complex. When determining the complexity of asegment, the system can also use the metadata associated with the mediaasset. For example, metadata related to a first segment may indicatethat the first segment has complex dialogue.

The system can distinguish between complex segments and interestingsegments. Sometimes users rewind a segment because the segment has a lotof action (e.g., interesting), not because the user is confused aboutthe complexity of the segment. For example, if the system uses viewinghabits to determine that a first segment is rewound often by users, thesystem can check the metadata of the first segment to determine if thefirst segment comprises action or dialogue. If the first segmentcomprises action, the first segment is likely an interesting segment,and if the first segment comprises dialogue the first segment is likelya complex scene. In another example, again the system may determine,using the viewing habits, that a first segment is rewound often byusers. The system may also determine the focal point and/or speed of thefirst segment using metadata of the first segment. If the firstsegment's focal point lacks stability and/or the speed of the scene isfast, the first segment is likely a complex segment, and if the firstsegment's focal point has stability and/or the speed of the scene isslow, the first segment is likely an interesting segment. The system canuse one or more of the complexity factors to determine if a firstsegment is complex. If the system determines that a first segment iscomplex, the system can tag the first segment with a complexityidentifier. The complexity factors may relate to a complexity score, andthe system may tag a segment with a complexity score over a giventhreshold with a complexity identifier.

The system can use the complexity factors along with user information togenerate supplemental content for a complex segment. For example, when auser initiates a trick play function during a first segment, the systemcan retrieve the complexity information (e.g., complexity factors)related to the first segment along with user information. The userinformation can relate to the user's demographic traits, the user'sgeneral knowledge of the information included in the first segment,information about the user's viewing environment, user's viewing habits,the user preferences for supplemental content, and similar suchinformation. For example, a user may rewind during a first segmenttagged as complex where the metadata associated with the first segmentindicates that the segment relates to a first topic (e.g., World War I).The system may determine that the user information indicates that theuser has never viewed any content relating to the first topic and thatthe first segment is tagged as complex so supplemental content isrequired. Accordingly, the system can generate supplemental content forthe user as the user rewinds the first segment. In another example, auser may rewind during a second segment tagged as complex where themetadata associated with the second segment indicates that the segmentrelates to a second topic (e.g., artificial intelligence). The systemmay determine that the user information indicates that the user oftenwatches content involving the second topic and determines that thesupplemental content is not required for the user. Accordingly, thesystem may rewind the first segment and not include any supplementalcontent. Sometimes the system may include the supplemental content ifthe user rewinds a scene more than once, or may present the user with aselectable option (e.g., “Would you like more information related tothis scene?”) to include supplemental comment.

The supplemental content may include any information that aids the userin understanding the complex segment. Supplemental content may includecontent featuring audio, video, and/or textual data associated with thecomplex segment. The supplemental content can be customized to the userbased on the received user information. For example, when a userinitiates a trick play function during a first segment, the userinformation can indicate that the user prefers supplemental content in avideo format, and the system can insert a video explaining the complexsegment for playback before resuming the complex segment. In anotherexample, the system may include a link to the video or overlay the videoonto the complex segment instead of presenting the video for playbackbefore resuming the complex segment. In another example, the userinformation may indicate that the user prefers supplemental content froma certain source (e.g., IMDb), and the system selects the highest-ratedvideo explaining the complex segment from the preferred source forplayback before resuming the complex segment.

The supplemental content may comprise a video montage generated by thesystem. The montage may be audio and/or video that incorporates previoussegments of the media asset. For example, if a complex segment relatesto a first and second segment that occurred earlier in the media asset,parts of the first and second segment can be included in the montage tohelp the user understand the complex segment. In another example, a userinitiates a trick play function during a complex segment of the mediaasset that is part of a series (e.g., television series, movie trilogy,etc.), and the generated montage includes segments from previouslypresented content. The supplemental content can be generated usingcomplexity factors and/or user information. For example, segments of amedia asset may be categorized based on content information (e.g.,actor, character, plot arc, topic, etc.). When the system determinesthat a first segment associated with a first content information (e.g.,plot arc dealing with revenge) is complex, the system can generate amontage including previous segments associated with the first contentinformation (e.g., previous actions inspiring the revenge plot arc).

BRIEF DESCRIPTION OF THE DRAWINGS

The below and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIGS. 1A-1C show illustrative diagrams of a user device includingsupplemental content along with complex media segments during trickplay, in accordance with embodiments of the disclosure;

FIGS. 2A and 2B show block diagrams of an illustrative process fordetermining that a segment of a media asset is complex, in accordancewith embodiments of the disclosure;

FIGS. 3A-3C show illustrative diagrams of a user device includingsupplemental content along with complex media segments during trickplay, in accordance with embodiments of the disclosure;

FIG. 4 shows an illustrative block diagram of a media system, inaccordance with embodiments of the disclosure;

FIG. 5 shows an illustrative block diagram of a user equipment (UE)device system, in accordance with embodiments of the disclosure;

FIG. 6 is an illustrative flowchart of a process for determining that asegment of a media asset is complex, in accordance with embodiments ofthe disclosure;

FIG. 7 is an illustrative flowchart of a process for generatingsupplemental content along with complex media segments during trickplay, in accordance with embodiments of the disclosure; and

FIG. 8 is another illustrative flowchart of a process for generatingsupplemental content along with complex media segments during trickplay, in accordance with embodiments of the disclosure.

DETAILED DESCRIPTION

FIGS. 1A-1C show illustrative diagrams of a user device 102 includingsupplemental content along with complex media segments during trickplay, in accordance with some embodiments of the disclosure. FIG. 1A'ssystem 100 comprises a user device 102 displaying a first scene 108 of amedia asset on a display 104. Although a television is used in thisexample, the user device 102 may be any device capable of playing amedia asset, such as televisions, laptops, tablets, smartphones, orsimilar such devices. The media asset/asset may be any type of mediaincluding television programs, movies, audio books, podcasts, playlists,songs, videos, audio, images or similar content, and any combinationthereof. In some embodiments, a media asset includes or is divided intoone or more segments, where the segments comprise one or more scenes.

FIG. 1B shows the user device 102 receiving a rewind command 114 from auser viewing a second scene 110. In some embodiments, the user inputsthe rewind command 114 by selecting a rewind button from a menu 106. Insome embodiments, the user inputs the rewind command 114 by pressing abutton on a remote control, issuing a voice command, and/or similar suchmethodologies. Although a rewind command 114 is displayed, similar suchcommands may be used.

In some embodiments, upon receiving the rewind command 114 the system100 determines whether the inclusion of supplemental content isrequired. In some embodiments, the system 100 provides supplementalcontent when receiving a trick play command (e.g., rewind command 114)during a complex segment. For this example, the first scene 108 and thesecond scene 110 are two scenes included in a first segment. In someembodiments, the system 100 uses complexity information about the firstsegment to determine if the first segment is complex. The complexityinformation may be included in metadata received by the user device 102.In some embodiments, the complexity information comprises a complexityidentifier and/or complexity factors. In some embodiments, complexityfactors include viewing habits of users who have watched the mediaasset, reviews related to the media asset, metadata associated with themedia asset, or similar such complexity-related information. Forexample, metadata associated with the first segment may contain anidentifier indicating that the first segment is complex if a firstnumber of users rewound the first segment of the media asset.

In some embodiments, if the system 100 determines that the first segmentis complex, the system 100 generates supplemental content 112. In someembodiments, supplemental content 112 may include audio, video, and/ortextual data associated with the first segment. FIG. 1C displays theuser device 102 displaying supplemental content 112 before resumingplayback of the first segment. In some embodiments, the system 100receives supplemental content related to the first segment from aserver. In some embodiments, the system 100 searches a database and/oronline platform for supplemental content related to the first segment.For example, the system 100 may search for reviews that discuss a wordor phrase (e.g., “No. I am your father.”) that is contained in thedialogue of the first segment (first scene 108). In another example, thesystem 100 may search for reviews associated with a first identifier(e.g., Luke's father) that is similar to or the same as a secondidentifier (e.g., Darth Vader) found in metadata associated with thefirst segment.

As shown in FIG. 1C, the user device 102 can display supplementalcontent 112 before resuming playback of the first segment. In someembodiments, the user device 102 rewinds the media asset to thebeginning of the first segment and plays supplemental content beforestarting playback of the first segment. In some embodiments, the userdevice 102 displays a selectable link to the supplemental content 112during the rewind operation 110 or overlays the supplemental content 112onto the first segment during playback.

In some embodiments, the supplemental content comprises a video montage.The montage may be a video that incorporates previous segments of themedia asset and/or related media assets. In some embodiments, segmentsand/or scenes of the media asset are categorized based on contentinformation (e.g., actor, character, plot arc, topic, etc.). In someembodiments, when the system 100 determines that the first segment,associated with a first content information item (e.g., Darth Vaderbeing Luke's father), is complex, the system generates a montageincluding previous segments that are also associated with the firstcontent information item (e.g., previous actions showing how Darth Vaderis Luke's father). For example, the video montage for the first segmentmay contain parts from a second segment (e.g., romance between Padmé andAnakin) and a third segment (e.g., Anakin becoming Darth Vader) becausethe second and third segments relate to the first segment (Darth Vaderbeing Luke's father) and help the user better understand the firstsegment.

The supplemental content 112 can be customized based on userinformation. In some embodiments, user information relates to the user'sdemographic traits, user's general knowledge on content information,information about the user's viewing environment, user's viewing habits,user preferences for supplemental content, and similar such information.In some embodiments, after receiving a trick play command (e.g., rewindcommand 114) during a complex segment, the system 100 uses the userinformation to generate the supplemental content 112. For example, theuser information may indicate that the user viewing the media assetprefers supplemental content 112 in a video format, and the system 100generates the supplemental content 112 in a video format. In anotherexample, the user information may indicate that the user viewing themedia asset prefers supplemental content 112 from a first source (e.g.,IMDb), and the system selects the highest-rated video relating to thefirst segment from the preferred source.

User information can also be used to determine if supplemental content112 should be generated and/or displayed. For example, the userinformation may indicate that a loud noise (e.g., doorbell, talking,etc.) was detected in the vicinity of the user device 102 before therewind command 114 was received. The system 100 may determine that nosupplemental content 112 is required, despite the first segment beingtagged as complex, because the loud noise was likely the reason for therewind command 114 and not the complexity of the scene. In anotherexample, the user information may indicate that the user often watchesmedia assets involving a first topic (e.g., space travel), and the userinputs a rewind command 114 during a first segment tagged as complex.The complexity information related to the first segment may indicatethat the first segment includes the first topic. The system 100 maydetermine that no supplemental content is required 112, despite thefirst segment being tagged as complex, because the user often watchescontent involving the first topic and likely issued the rewind command114 due to interest in the first segment rather than being confused bythe first segment. In another example, the user information may indicatethat the user never watches media assets involving a second topic (e.g.,Christmas), and the user inputs a rewind command 114 during a firstsegment not tagged as complex. The complexity information related to thefirst segment may indicate that the first segment includes the secondtopic but has a low complexity score. The system 100 may determine thatsupplemental content is required 112 despite the first segment not beingtagged as complex because the user never watches content involving thesecond topic and likely issued the rewind command 114 due to confusionwith the first segment rather than being interested or amused by thefirst segment.

FIGS. 2A and 2B show block diagrams of an illustrative process fordetermining that a segment of a media asset is complex, in accordancewith some embodiments of the disclosure. In some embodiments, thecomplexity module 202 receives consumption data 204 of a media asset,which is collected by monitoring consumption of the media asset. In someembodiments, the consumption data 204 may be collected at or by thecomplexity module 202. In some embodiments, the consumption data 204includes trick play commands received during consumption of the mediaasset. For example, in FIG. 2B, a first consumption data 214 includesthe number of users that rewound and paused a plurality of segments ofthe media asset. As shown, the first consumption data 214 includes thedata related to all segments that were rewound or paused. In someembodiments, the first consumption data 214 includes only segments thatreceived at least a threshold number of trick play commands.

In some embodiments, the complexity module 202 receives reviews 206relating to the media asset. In some embodiments, the reviews 206 may becollected at or by the complexity module 202. In some embodiments, thecomplexity module 202 receives reviews 206 related to the media assetfrom a server. In some embodiments, the complexity module 202 searches adatabase and/or online platform for reviews 206 related to the mediaasset. For example, the complexity module 202 may search for reviewsthat discuss a word or phrase (e.g., “No. I am your father.”) that iscontained in the dialogue of the media asset. In another example, thecomplexity module 202 may search for reviews associated with a firstidentifier (e.g., Luke's father) that is similar or the same to a secondidentifier (e.g., Darth Vader) found in metadata associated with themedia asset and/or a segment of the media asset. In some embodiments,the reviews 206 include review information, such as an indicator that areview relates to a first segment of the media asset. For example, asshown in FIG. 2B, the review information 216 can include the segmentthat a review relates to, the review ID, the format of a review, theduration of a review, and the source of a review. Although the reviews206 are described as a plurality of reviews, in some embodiments thereviews 206 are a single review. In some embodiments, the complexitymodule 202 filters the reviews 206. For example, the complexity module202 may check a first review's credentials (e.g., number of views,number of likes, number of comments, number of verifications, etc.) anddetermine if the credentials satisfy certain parameters. In someembodiments, the complexity module 202 only uses reviews 206 thatsatisfy a first parameter (e.g., more than 1,000 views).

In some embodiments, the complexity module 202 receives the media asset208. In some embodiments, the media asset 208 also includes metadatarelated to the media asset 208. In some embodiments, the metadataincludes content information (e.g., actor, character, plot arc, topic,etc.), audio information (e.g., dialogue, music, etc.), videoinformation (e.g., focal point of a segment, speed of a segment, etc.),and/or similar such information. For example, as shown in FIG. 2B, themedia asset 208 can comprise a first set of metadata 218 related to themedia asset, including segment identifiers, topics related to thesegments, and dialogue indicators.

In some embodiments, the complexity module 202 uses the consumption data204, reviews 206, and/or the media asset 208 to determine if one or moresegments of the media asset 208 are complex. In some embodiments, thecomplexity module 202 can use a single input to determine if one or moresegments of the media asset 208 are complex. For example, if the firstconsumption data 214 indicates that a segment (e.g., segment 12) hasbeen rewound by a threshold number of users (e.g., more than 300 users),then the complexity module 202 can determine that the segment (e.g.,segment 12) is a complex segment. In another example, if the reviewinformation 216 indicates that one or more reviews (e.g., review ID 251and review ID 252) relate to a segment (e.g., segment 12), then thecomplexity module 202 can determine that the segment (e.g., segment 12)is a complex segment. In another example, if a first set of metadata 218indicates that a segment (e.g., segment 12) involves more than one topicand/or involves a complex topic, then the complexity module 202 candetermine that the segment (e.g., segment 12) is a complex segment.

In some embodiments, the complexity module 202 can use multiple inputsto determine if one or more segments of the media asset 208 are complex.For example, the complexity module 202 may receive consumption data 204indicating that a first segment is rewound by a number of users, wherethe number exceeds a first threshold. The complexity module 202 may alsouse metadata of the received media asset 208 to determine that the firstsegment involves no dialogue. In some embodiments, the complexity module202 uses the number of users who rewound the first segment and themetadata indicating that there was no dialogue in the first segment todetermine that the first segment is not complex. In some embodiments,the complexity module 202 makes this determination because a segmentthat is rewound more than the first threshold but has no dialogue islikely an action sequence and is unlikely to be a complex segment. Insome embodiments, the complexity module 202 uses one or more conditionsto determine if a segment is complex. For example, a first condition maybe that if the number of users who rewound a first segment is above thefirst threshold and the metadata indicates that there was dialogue inthe first segment, the first segment is complex. In some embodiments,the complexity module uses the received inputs (e.g., consumption data204, reviews 206, and media asset 208), to generate a complexity score.In some embodiments, the different inputs are weighted differently. Forexample, consumption data 204 indicating that a first number of users(e.g., 500 users) rewound a first segment may be weighted higher thanreview information indicating that there are only a few reviews relatingto the first segment. In another example, consumption data 204indicating that a first number of users (e.g., 100 users) rewound afirst segment may be weighted less than if the consumption data 204indicated that a second number of users (e.g., 500 users) rewound thefirst segment.

In some embodiments, each segment of the media asset 208 that thecomplexity module 202 determined to be complex is tagged with acomplexity identifier, generating the media asset with complexityidentifiers 210. In some embodiments, the media asset with complexityidentifiers 210 is transmitted to user devices (e.g., user device 102)for playback. In some embodiments, the complexity identifiers areincluded in the metadata of the media asset with complexity identifiers210. FIG. 2B displays the first consumption data 214, the reviewinformation 216, and a first set of metadata 218 having the same segmentidentifiers (e.g., segment 12) corresponding to the same segments of themedia asset. In some embodiments, segments are defined by the complexitymodule 202. The complexity module 202 may define complex segments bydetermining the start and end times of complex segments using one ormore received inputs (e.g., consumption data 204, reviews 206, mediaasset 208, etc.). For example, the metadata related to the media assetmay indicate that a scene starting at a first time and ending at secondtime corresponds to a first topic (e.g., time travel). In someembodiments, if the complexity module 202 determines that the firsttopic is complex, then the complexity module defines the portion of themedia asset between the start and end time of the first topic as acomplex segment.

In some embodiments, the complexity module 202 uses the consumption data204, reviews 206, and/or the media asset 208 to generate a knowledgegraph for the media asset 208. In some embodiments, the generation ofthe knowledge graph uses semantic enrichment processes to constructnodes, labels, etc., and the complexity module 202 can use the knowledgegraph to determine whether to tag a segment as complex and/or generatesupplemental content. In some embodiments, the complexity module 202uses the nodes of the knowledge graph to determine the complexity scoreof content information (e.g., actor, character, plot arc, topic, etc.)and/or a segment of the media asset 208 based on the number of links tothe nodes corresponding to said information. In some embodiments, thecomplexity score is determined based on in intrinsic data (data aboutthe segment being viewed/info in the knowledge graph) and/or other data.For example, the other data may be associated with local sensors in theuser's vicinity (e.g., motion sensors, noise sensors, etc.) as well asprofile data of the viewer. In some embodiments, sensor data is anindication that a received trick play command is in response to adistraction and is not in response to a poor understanding of the topic.

In some embodiments, replaying a political-segment or a segment whichrelates to historical events might result in the extraction andpresentation of the main topics in the segment, related topics insubsequent segments, and/or subsequent episodes of the media asset(i.e., segments that will be presented later in the media asset in orderto prevent another a trick play command and/or viewing interruption). Insome embodiments, the system displays multiple options for supplementalcontent where each option corresponds to a different topic and the usercan select the option corresponding to the topic that they want to seesupplemental information about. For example, a segment from a movie thattalks about the “prohibition period” in the early 1900s may result inthe display of selectable options for ‘prohibition’ and ‘bootlegging’ if“bootlegging” is discussed in a subsequent segment of the media asset.In some embodiments, supplemental content can be in any form (text,pictures, short videos, etc.) and obtained from various sources (e.g.,verified websites, search engines, and/or a predefined pool ofcontent/content sources). In some embodiments, a generated knowledgegraph for the media asset is used to determine similar/related topics orconcepts. In some embodiments, a knowledge graph for the media asset ispartially constructed based on a more general knowledge graph that isnot specific to the media asset.

FIGS. 3A-3C show illustrative diagrams of a user device 302 includingsupplemental content along with a complex segment 308, in accordancewith some embodiments of the disclosure. FIG. 3A displays a user device302 displaying a complex segment 308 of a media asset on a display 304.In some embodiments, the user device 302 is resuming playback 310 of acomplex segment 308 after a user issued a first trick play command(e.g., pause, rewind, etc.). In some embodiments, the user device 302resumes playback 310 after rewinding to the beginning of the complexsegment 308. In some embodiments, the user device 302 resumes playback310 after receiving a play command from a user. In some embodiments, auser inputs one or more commands using a selectable menu 306. In someembodiments, the user device 302 uses any of the methodologies describedherein to determine that the complex segment 308 is complex. Forexample, the user device 302 may determine that the complex segment 308is complex because the metadata associated with the complex segment 308comprises a complexity identifier.

In some embodiments, the user device 302 includes a first option 312along with the complex segment 308. In some embodiments, the firstoption 312 is displayed if the user device 302 has access tosupplemental content. In some embodiments, the first option 312 isdisplayed if a cloud-service that is responsible for aggregatingadditional content about the segment, has such content available. Insome embodiments, the first option 312 is overlaid on the complexsegment 308 during playback 310. Upon selection of the first option 312,the user device 302 can display supplemental content. For example, theuser device 302 may display a review related to the complex segment 308as an overlay, as the complex segment 308 continues playing. The userdevice 302 may display the first option 312 for certain types of complexsegments and/or for certain instances. For example, the first option 312may be available for segments geared toward children and/or educationalsegments. In another example, the first option 312 may be generatedafter the user causes the user device 302 to rewind the complex segment308 more than once. In some embodiments, the user device 302 uses any ofthe methodologies described herein to generate and/or displaysupplemental content.

In some embodiments, certain supplemental content may be designed to beplayed along with the complex segment 308. For example, the supplementalcontent may include visual overlays (e.g., boxes, pointers, etc.) tohelp direct the user's attention to important elements of the complexsegment 308. In another example, the supplemental content may include areview explaining important elements of the complex segment 308 as thecomplex segment 308 plays. In another example, if the system determinesthat the user is reviewing a segment to find a hidden feature (e.g.,Easter egg) the supplemental content may include visual overlays to helpdirect the user's attention to the hidden feature. In some embodiments,the system uses metadata related to the segment to determine a locationof a hidden feature.

As shown in FIG. 3B, the user device 302 can include a second option 314along with the complex segment 308. In some embodiments, the secondoption 314 is a link to supplemental content. In some embodiments, thesecond option 314 is overlaid on the complex segment 308 during playback310. Upon selection of the second option 314, the user device 302 candisplay supplemental content. For example, the user device 302 maydisplay a review related to the complex segment 308 as an overlay as thecomplex segment 308 continues playing. In another example, the userdevice 302 may pause playback of the complex segment 308 and display areview related to the complex segment 308 before resuming playback. Insome embodiments, the second option 314 includes a thumbnail relating tothe supplemental content. For example, the second option 314 may includea thumbnail of a review of the complex segment 308. In some embodiments,the user device 302 uses any of the methodologies described herein togenerate and/or display supplemental content.

As shown in FIG. 3C, the user device 302 can include a firstsupplemental content 316 along with the complex segment 308. In someembodiments, the first supplemental content 316 is text, audio, and/orvideo overlaid on the complex segment 308 during playback 310. In someembodiments, the first supplemental content 316 is played automaticallyupon a user issuing a trick play command during the complex segment 308.In some embodiments, the user device 302 uses any of the methodologiesdescribed herein to generate and/or display the first supplementalcontent 316. In some embodiments, the user device 302 determines todisplay the first option 312, second option 314, and/or firstsupplemental content based on user information. For example, the userinformation may indicate that the preference for supplemental content ofthe user viewing the complex segment 308 is text supplemental content(e.g., first supplemental content 316) overlaid during playback of acomplex segment. In another example, the user information may indicatethat the user viewing the complex segment 308 prefers a selectableobject (e.g., first option 312, second option 314) that causes playbackof supplemental content.

FIGS. 4-5 describe exemplary devices, systems, servers, and relatedhardware for including supplemental content along with complex mediasegments during trick play, in accordance with some embodiments of theenclosure. In the system 400, there can be more than one user equipmentdevice 402, but only one is shown in FIG. 4 to avoid overcomplicatingthe drawing. In addition, a user may utilize more than one type of userequipment device and more than one of each type of user equipmentdevice. In an embodiment, there may be paths between user equipmentdevices, so that the devices may communicate directly with each othervia communications paths, as well as other short-range point-to-pointcommunications paths, such as USB cables, IEEE 1394 cables, wirelesspaths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or othershort-range communication via wired or wireless paths. BLUETOOTH is acertification mark owned by Bluetooth SIG, INC. In an embodiment, theuser equipment devices may also communicate with each other directlythrough an indirect path via the communications network 406.

The user equipment devices may be coupled to communications network 406.Namely, the user equipment device 402 is coupled to the communicationsnetwork 406 via communications path 404. The communications network 406may be one or more networks including the Internet, a mobile phonenetwork, mobile voice or data network (e.g., a 4G or LTE network), cablenetwork, public switched telephone network, or other types ofcommunications network or combinations of communications networks. Thepath 404 may separately or in together with other paths include one ormore communications paths, such as, a satellite path, a fiber-opticpath, a cable path, a path that supports Internet communications (e.g.,IPTV), free-space connections (e.g., for broadcast or other wirelesssignals), or any other suitable wired or wireless communications path orcombination of such paths. In one embodiment, path 404 can be a wirelesspath. Communications with the user equipment device 402 may be providedby one or more communications paths but is shown as a single path inFIG. 4 to avoid overcomplicating the drawing.

The system 400 also includes media content source 412, and server 414,which can be coupled to any number of databases providing information tothe user equipment devices. The media content source 412 represents anycomputer-accessible source of content, such as a storage for audiocontent, metadata, or, similar such information. The server 414 maystore and execute various software modules to implement the providing ofsupplemental content along with complex media segments functionality. Insome embodiments, the user equipment device 402, media content source412, and server 414 may store metadata associated with a media asset.

FIG. 5 shows a generalized embodiment of a user equipment device 500, inaccordance with one embodiment. In an embodiment, the user equipmentdevice 500 is the same user equipment device 402 of FIG. 4 . The userequipment device 500 may receive content and data via input/output(hereinafter “I/O”) path 502. The I/O path 502 may provide audio content(e.g., broadcast programming, on-demand programming, Internet content,content available over a local area network (LAN) or wide area network(WAN), and/or other content) and data to control circuitry 504, whichincludes processing circuitry 506 and a storage 508. The controlcircuitry 504 may be used to send and receive commands, requests, andother suitable data using the I/O path 502. The I/O path 502 may connectthe control circuitry 504 (and specifically the processing circuitry506) to one or more communications paths. I/O functions may be providedby one or more of these communications paths but are shown as a singlepath in FIG. 5 to avoid overcomplicating the drawing.

The control circuitry 504 may be based on any suitable processingcircuitry such as the processing circuitry 506. As referred to herein,processing circuitry 506 should be understood to mean circuitry based onone or more microprocessors, microcontrollers, digital signalprocessors, programmable logic devices, field-programmable gate arrays(FPGAs), application-specific integrated circuits (ASICs), etc., and mayinclude a multi-core processor (e.g., dual-core, quad-core, hexa-core,or any suitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiple of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). The providing of supplemental content along with complexmedia segments functionality can be at least partially implemented usingthe control circuitry 504. The providing of supplemental content alongwith complex media segments functionality described herein may beimplemented in or supported by any suitable software, hardware, orcombination thereof. The providing of supplemental content along withcomplex media segments functionality can be implemented on userequipment, on remote servers, or across both.

In client/server-based embodiments, the control circuitry 504 mayinclude communications circuitry suitable for communicating with one ormore servers that may at least implement the described providing ofsupplemental content along with complex media segments functionality.The instructions for carrying out the above-mentioned functionality maybe stored on the one or more servers. Communications circuitry mayinclude a cable modem, an integrated service digital network (ISDN)modem, a digital subscriber line (DSL) modem, a telephone modem, anEthernet card, or a wireless modem for communications with otherequipment, or any other suitable communications circuitry. Suchcommunications may involve the Internet or any other suitablecommunications networks or paths. In addition, communications circuitrymay include circuitry that enables peer-to-peer communication of userequipment devices, or communication of user equipment devices inlocations remote from each other (described in more detail below).

Memory may be an electronic storage device provided as the storage 508that is part of the control circuitry 504. As referred to herein, thephrase “electronic storage device” or “storage device” should beunderstood to mean any device for storing electronic data, computersoftware, or firmware, such as random-access memory, read-only memory,hard drives, optical drives, digital video disc (DVD) recorders, compactdisc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D discrecorders, digital video recorders (DVRs, sometimes called a personalvideo recorders, or PVRs), solid-state devices, quantum storage devices,gaming consoles, gaming media, or any other suitable fixed or removablestorage devices, and/or any combination of the same. The storage 508 maybe used to store various types of content described herein. Nonvolatilememory may also be used (e.g., to launch a boot-up routine and otherinstructions). Cloud-based storage, described in relation to FIG. 4 ,may be used to supplement the storage 508 or instead of the storage 508.

The control circuitry 504 may include audio generating circuitry andtuning circuitry, such as one or more analog tuners, audio generationcircuitry, filters or any other suitable tuning or audio circuits orcombinations of such circuits. The control circuitry 504 may alsoinclude scaler circuitry for upconverting and down converting contentinto the preferred output format of the user equipment device 500. Thecontrol circuitry 504 may also include digital-to-analog convertercircuitry and analog-to-digital converter circuitry for convertingbetween digital and analog signals. The tuning and encoding circuitrymay be used by the user equipment device 500 to receive and to display,to play, or to record content. The circuitry described herein,including, for example, the tuning, audio generating, encoding,decoding, encrypting, decrypting, scaler, and analog/digital circuitry,may be implemented using software running on one or more general purposeor specialized processors. If the storage 508 is provided as a separatedevice from the user equipment device 500, the tuning and encodingcircuitry (including multiple tuners) may be associated with the storage508.

The user may utter instructions to the control circuitry 504, which arereceived by the microphone 516. The microphone 516 may be any microphone(or microphones) capable of detecting human speech. The microphone 516is connected to the processing circuitry 506 to transmit detected voicecommands and other speech thereto for processing.

The user equipment device 500 may optionally include an interface 510.The interface 510 may be any suitable user interface, such as a remotecontrol, mouse, trackball, keypad, keyboard, touchscreen, touchpad,stylus input, joystick, or other user input interfaces. A display 512may be provided as a stand-alone device or integrated with otherelements of the user equipment device 500. For example, the display 512may be a touchscreen or touch-sensitive display. In such circumstances,the interface 510 may be integrated with or combined with the microphone516. When the interface 510 is configured with a screen, such a screenmay be one or more of a monitor, a television, a liquid crystal display(LCD) for a mobile device, active matrix display, cathode ray tubedisplay, light-emitting diode display, organic light-emitting diodedisplay, quantum dot display, or any other suitable equipment fordisplaying visual images. In some embodiments, the interface 510 may beHDTV-capable. In some embodiments, the display 512 may be a 3D display.The speaker (or speakers) 514 may be provided as integrated with otherelements of user equipment device 500 or may be a stand-alone unit.

The user equipment device 500 of FIG. 5 can be implemented in system 400of FIG. 4 as user equipment device 402, but any other type of userequipment suitable for providing of supplemental content along withcomplex media segments may be used. For example, user equipment devicessuch as television equipment, computer equipment, wireless usercommunication devices, or similar such devices may be used. Userequipment devices may be part of a network of devices. Various networkconfigurations of devices may be implemented and are discussed in moredetail below.

FIG. 6 is an illustrative flowchart of a process 600 for determiningthat a segment of a media asset is complex, in accordance withembodiments of the disclosure. Process 600, and any of the followingprocesses, may be executed by control circuitry 504 on a user equipmentdevice 500. In some embodiments, control circuitry 504 may be part of aremote server separated from the user equipment device 500 by way of acommunications network or distributed over a combination of both. Insome embodiments, instructions for executing process 600 may be encodedonto a non-transitory storage medium (e.g., the storage 508) as a set ofinstructions to be decoded and executed by processing circuitry (e.g.,the processing circuitry 506). Processing circuitry may, in turn,provide instructions to other sub-circuits contained within controlcircuitry 504, such as the encoding, decoding, encrypting, decrypting,scaling, analog/digital conversion circuitry, and the like. It should benoted that the process 600, or any step thereof, could be performed on,or provided by, any of the devices shown in FIGS. 1-5 . Although theprocess 600 is illustrated as described as a sequence of steps, it iscontemplated that various embodiments of process 600 may be performed inany order or combination and need not include all the illustrated steps.

At 602, control circuitry monitors a plurality of users viewing a mediaasset, wherein the media asset comprises a plurality of segments. Insome embodiments, the control circuitry monitors the plurality of users'viewing habits to determine consumption data. For example, the controlcircuitry may provide (e.g., transmit) the media asset to a user andmonitor whenever a user issues a trick play command. In someembodiments, whenever a user issues a trick play command, the controlcircuitry is notified. In some embodiments, a second device records eachtrick play command received from a user watching the media asset andtransmits the aggregated data to the control circuitry. In someembodiments, the control circuitry monitors the number of users whorewound and/or paused a plurality of segments of the media asset. Thecontrol circuitry may also record additional information (e.g.,demographic traits of the user, time of trick play command, etc.)related to the trick play command. In some embodiments, the controlcircuitry stores the consumption data in a database. In someembodiments, the control circuitry only stores consumption data forsegments that receive a threshold number of trick play commands.

At 604, control circuitry determines a number of users who rewound afirst segment of the media asset during playback of the media asset. Insome embodiments, the control circuitry uses consumption data generatedduring the monitoring of step 602 to determine the number of users whorewound the first segment. In some embodiments, the control circuitrystores and/or receives the consumption data in a table format (e.g.,first consumption data 214). In some embodiments, the control circuitrycan use the consumption data to determine trends. For example, thecontrol circuitry can determine that a first group of users issued arewind command during a first segment and that a majority of the firstgroup of users have a first demographic trait (e.g., age, gender,ethnicity, location, occupation, etc.). In some embodiments, the controlcircuitry stores the trends in a database.

At 606, control circuitry receives one or more reviews relating to thefirst segment of the media asset. In some embodiments, the controlcircuitry receives a plurality of reviews related to the media assetfrom a server. In some embodiments, the control circuitry requestsand/or searches for reviews relating to the first segment and/or thesegments during which the plurality of users issued trick play commands.In some embodiments, the control circuitry searches a database and/oronline platform for reviews related to the first segment and/or mediaasset. For example, the control circuitry may search for reviews thatdiscuss a word or phrase (e.g., “No. I am your father.”) that iscontained in the dialogue of the first segment. In another example, thecontrol circuitry may search for reviews associated with a firstidentifier (e.g., Luke's father) that is similar to or the same as asecond identifier (e.g., Darth Vader) found in metadata associated withthe media asset and/or the first segment of the media asset. In someembodiments, the one or more reviews include review information, such asan indicator that a review relates to the media asset and/or the firstsegment of the media asset. For example, the review information caninclude the segment that a review relates to, the review ID, the formatof a review, the duration of a review, and the source of a review.

At 608, the control circuitry determines whether the number of users whorewound the first segment is over a first threshold. In someembodiments, the first threshold is based on the total number of usersmonitored in step 602. For example, if 100 users were monitored, thefirst threshold may be ten, and if 1000 users were monitored, the firstthreshold may be 100. If the control circuitry determines that thenumber of users who rewound the first segment is over a first threshold,the process 600 continues to step 610. If the control circuitrydetermines the number of users who rewound the first segment is not overthe first threshold, the process 600 continues back to step 602.

At 610, the control circuitry determines whether one or more reviewsrelating to the first segment were received. If the control circuitrydetermines that one or more reviews relating to the first segment werereceived, the process 600 continues to step 612. If the controlcircuitry determines one or more reviews relating to the first segmentwere not received, the process 600 continues back to step 602. In someembodiments, the process 600 continues to step 612 only if the controlcircuitry determines that a second threshold number of reviews arereceived. In some embodiments, the process 600 continues to step 612only if the control circuitry determines that one or more reviewssatisfy one or more parameters (e.g., more than 1,000 views). In someembodiments, the control circuitry uses credentials (e.g., number ofviews, number of likes, number of comments, number of verifications,etc.) of the one or more reviews to determine if the one or moreparameters are satisfied.

At 612, the control circuitry tags the first segment with an identifierindicating that the first segment is a complex segment. In someembodiments, the identifier is included in the metadata of the mediaasset and/or first segment. In some embodiments, the identifier includesone or more complexity factors (e.g., the viewing habits of users whohave watched the first segment, reviews related to the first segment,complexity score for the first segment, etc.). In some embodiments, eachsegment of the plurality of segments that the control circuitrydetermines to be complex is tagged with a complexity identifier,generating a media asset with complexity identifiers. In someembodiments, the media asset with complexity identifiers is transmittedto one or more user devices for playback. In some embodiments, theidentifier indicating that the first segment is complex comprisesdemographic information related to the number of users who rewound thefirst segment. For example, if a majority of the number of users whorewound the first segment have a first demographic trait (e.g., undertwelve years old) then the identifier may include the demographic traitin the identifier. Accordingly, a user device may determine that thefirst segment is complex for users of the first demographic trait (e.g.,under twelve years old) but not for users of a second demographic trait(e.g., over twelve years old).

FIG. 7 is an illustrative flowchart of a process 700 for generatingsupplemental content along with complex media segments during trickplay, in accordance with embodiments of the disclosure.

At 702, control circuitry receives a media asset comprising a pluralityof segments associated with metadata. In some embodiments, the mediaasset is received from a server. In some embodiments, the metadata isreceived with the media asset. In some embodiments, the metadataincludes content information (e.g., actor, character, plot arc, topic,etc.), audio information (e.g., dialogue, music, etc.), videoinformation (e.g., focal point of a segment, speed of a segment, etc.),identifiers (e.g., complexity identifier) and/or similar suchinformation.

At 704, control circuitry receives a command from a user to rewind thefirst segment. In some embodiments, the command 110 is received when auser selects a rewind button from a menu (e.g., menu 106) during thefirst segment. In some embodiments, the rewind command is received whena user presses a button on a remote control, issues a voice commandand/or similar such methodologies. Although a rewind command isdiscussed in process 700, similar such commands may be used (e.g.,pause, skip, fast-forward, etc.) and supplemental content can begenerated according to process 700.

At 706, control circuitry determines whether the metadata related to thefirst segment comprises a complexity identifier. In some embodiments,the complexity identifier includes one or more complexity factors (e.g.,the viewing habits of users who have watched the first segment, reviewsrelated to the first segment, complexity score for the first segment,etc.). In some embodiments, the complexity identifier comprisesdemographic information related to a number of users who rewound thefirst segment. If the metadata related to the first segment comprises acomplexity identifier, the process 700 continues to step 708. If themetadata related to the first segment does not comprise a complexityidentifier, the process 700 continues to step 710. At step 710, controlcircuitry rewinds the first segment without providing supplementalcontent. In some embodiments, the control circuitry monitors the user'sviewing behavior to update a profile associated with the user. Forexample, the control circuitry may record that the user issued thecommand received in step 704 during the first segment.

At 708, control circuitry receives user information related to a profileassociated with the user. In some embodiments, the profile stores userinformation relating to the user's demographic traits, viewingenvironment, viewing habits, preferences, and similar such information.

At 712, control circuitry receives supplemental content related to thefirst segment. In some embodiments, the control circuitry receives aplurality of supplemental content related to the media asset from aserver, wherein the plurality of supplemental content comprisessupplemental content related to the first segment and/or the mediaasset. In some embodiments, the control circuitry requests and/orsearches for supplemental content (e.g., reviews) relating to the firstsegment and/or media asset. In some embodiments, the control circuitrysearches a database and/or online platform for supplemental contentrelated to the first segment and/or media asset. For example, thecontrol circuitry may search for reviews that discuss a word or phrase(e.g., “No. I am your father.”) that is contained in the dialogue of thefirst segment. In another example, the control circuitry may search forsupplemental content associated with a first identifier (e.g., Luke'sfather) that is similar to or the same as a second identifier (e.g.,Darth Vader) found in metadata associated with the media asset and/orthe first segment of the media asset. In some embodiments, one or morepieces of the received supplemental content indicate supplementalcontent information (e.g., the segment to which the supplemental contentrelates, the supplemental content ID, the format of the supplementalcontent, the duration of the supplemental content, and the source of thesupplemental content).

At 714, control circuitry generates a first piece of supplementalcontent to display using the user information. In some embodiments, thecontrol circuitry selects the first piece of supplemental content fromthe supplemental content received in step 712. In some embodiments, thecontrol circuitry generates the first piece of supplemental contentbased on user preferences. For example, the user information canindicate that the user prefers supplemental content in an audio format,and the control circuitry can select an audio review explaining thefirst segment from the supplemental content received in step 712. Inanother example, the user information can indicate that the user preferssupplemental content from a certain source (e.g., IMDb), and the controlcircuitry can select the highest-rated video relating to the firstsegment from the preferred source as the first piece of supplementalcontent. In some embodiments, the control circuitry determines thedisplay format of the generated first piece of supplemental contentbased on user information. For example, the user information mayindicate that the user prefers supplemental content as textualsupplemental content overlaid during playback of the first segment. Inanother example, the user information may indicate that the user prefersa selectable object that, when selected, causes playback of the firstpiece of supplemental content.

In some embodiments, the control circuitry generates the first piece ofsupplemental content using the received supplemental content. Forexample, the control circuitry may generate a video montage (first pieceof supplemental content) using previous segments of the media assetand/or related media assets (received supplemental content). In someembodiments, segments and/or scenes of the media asset are categorizedbased on content information (e.g., actor, character, plot arc, topic,etc.). In some embodiments, when the control circuitry determines thatthe first segment is associated with a first content information (e.g.,Darth Vader being Luke's father) the control circuitry generates amontage including previous segments that are also associated with thefirst content information (e.g., previous actions showing how DarthVader is Luke's father). For example, the video montage for the firstsegment may contain parts from a second segment (e.g., romance betweenPadmé and Anakin) and a third segment (e.g., Anakin becoming DarthVader) because the second and third segments relate to the first segment(Darth Vader being Luke's father) and help the user better understandthe first segment.

In some embodiments, when a media asset is broadcasted, the controlcircuitry associates key segments/scenes of the media asset with aspecific actor (e.g., unique ID). In some embodiments, the controlcircuitry categorizes the segments based on their content (e.g., goodtimes, revenge, etc.). In some embodiments, categorizing the segmentsbased on their content allows the control circuitry to tie an actiontaken by a character to a segment. In some embodiments, the controlcircuitry updates a data structure(s) associated with a character assubsequent episodes, related to the media asset, are broadcast. Suchdata structures can reference specific segments which may be accessibleto a montage creation service that stitches segments from differentepisodes together or selects a few segments to stitch together based onthe length of the desired montage. For example, a first user might enjoywatching a short montage (e.g., thirty seconds) while a second usermight prefer a longer montage (e.g., two minutes). In some embodiments,the montage service can create and update montages for different viewers(after more episodes related to the media asset are watched and/or moreof the media asset is consumed). In some embodiments, the montageservice utilizes a user's current viewing progress to dynamicallygenerate such montages, preventing the showing of spoilers to the user.In some embodiments, the availability or option to present the montagecan be tied to specific portions of the content and/or is presented inresponse to a general user query such as “what did he mean?”

FIG. 8 is another illustrative flowchart of a process for generatingsupplemental content along with complex media segments during trickplay, in accordance with embodiments of the disclosure.

At 802, control circuitry receives a command from a user to rewind afirst segment of a media asset during playback. In some embodiments,step 802 uses the same or similar methodologies as step 704 describedabove.

At 804, control circuitry determines whether the first segment comprisesa complexity identifier. In some embodiments, the control circuitry usesmetadata associated with the first segment to determine if the firstsegment comprises a complexity identifier. In some embodiments, thecomplexity identifier includes one or more complexity factors (e.g., theviewing habits of users who have watched the first segment, reviewsrelated to the first segment, complexity score for the first segment,etc.). In some embodiments, the complexity identifier comprisesdemographic information related to a number of users who rewound thefirst segment. If the control circuitry determines that the firstsegment comprises a complexity identifier, the process 800 continues tostep 806. If the control circuitry determines the first segment does notcomprise a complexity identifier, the process 800 continues to step 808.At step 808, control circuitry rewinds the first segment withoutproviding supplemental content. In some embodiments, the controlcircuitry monitors the user's viewing behavior to update a profileassociated with the user.

At 806, control circuitry receives user information related to a profileassociated with the user. In some embodiments, the profile stores userinformation relating to the user's demographic traits, viewingenvironment, viewing habits, preferences, and/or similar suchinformation. In some embodiments, the control circuitry requests userinformation related to the first segment. For example, if the firstsegment is associated with a first topic (e.g., World War I) the controlcircuitry may request user information related to the user's viewinghistory of media assets relating to the first topic. In another example,if the first segment is associated with age-specific information (e.g.,reference to a movie from 1982), the control circuitry may request userinformation related to the user's age. In another example, if the firstsegment is associated with a location (e.g., New York) the controlcircuitry may request user information related to the user's location.

At 810, control circuitry determines whether supplemental content isneeded. In some embodiments, the control circuitry determines whethersupplemental content is needed using a complexity identifier. Forexample, if the first segment comprises a complexity identifier, thenthe control circuitry may determine that supplemental content isrequired.

In some embodiments, the control circuitry uses the received userinformation to determine if supplemental content is needed. For example,the user information may indicate that a loud noise (e.g., doorbell,talking, etc.) was detected in the vicinity of the user device beforethe received command was received. The control circuitry may determinethat no supplemental content is needed despite the first segmentcomprising a complexity identifier because the loud noise was likely thereason for the received command and not the complexity of the scene. Inanother example, the control circuitry may receive the command from theuser during the first segment, tagged as complex, and the userinformation may indicate that the user often watches media assetsinvolving a first topic (e.g., space travel). The control circuitry maydetermine (e.g., using complexity information related to the firstsegment) that the first segment includes the first topic. The controlcircuitry may determine that no supplemental content is needed, despitethe first segment being tagged as complex, because the user oftenwatches content involving the first topic and likely issued the commanddue to interest in the first segment rather than being confused by thefirst segment. In another example, the control circuitry may receive thecommand from the user during the first segment, tagged as complex, andthe user information may indicate that the user has a first demographictrait (e.g., being ten years old). The control circuitry may determine(e.g., using complexity information related to the first segment) thatthe first segment is often rewound by a first group of users with ashared demographic trait (e.g., users less than 15 years old). Thecontrol circuitry may determine that supplemental content is neededbecause the user's first demographic trait falls within the shareddemographic trait. In another example, the control circuitry may receivethe command from the user during the first segment tagged as complex andthe user information may indicate that the user has a first demographictrait (e.g., being 20 old). The control circuitry may determine (e.g.,using complexity information related to the first segment) that thefirst segment is often rewound by a first group of users with a shareddemographic trait (e.g., users less than 15 years old). The controlcircuitry may determine that supplemental content is not needed becausethe user's first demographic trait falls outside of the shareddemographic trait.

In some embodiments, the control circuitry uses the complexityinformation and/or user information to generate a complexity score. Insome embodiments, if the control circuitry determines that thecomplexity score is over a threshold, the control circuitry determinesthat supplemental content is needed. In some embodiments, the controlcircuitry weights inputs differently when generating a complexity score.For example, consumption data indicating that a first segment wasrewound by a first number of users (e.g., 500 users) may be weightedhigher than review information indicating that there are only a fewreviews relating to the first segment. In another example, the userinformation may indicate that a loud noise (e.g., doorbell, talking,etc.) was detected in the vicinity of the user device before thereceived command was received, and the complexity information mayindicate that a first number of users (e.g., 500 users) rewound thefirst segment. In some embodiments, the control circuitry weights theuser information higher than the complexity information and determinesthat no supplemental content is needed. If the control circuitrydetermines that supplemental content is needed, the process 800continues to step 812. If the control circuitry determines that nosupplemental content is needed, the process 800 continues to step 808.

At 812, control circuitry receives supplemental content related to thefirst segment. In some embodiments, step 812 uses the same or similarmethodologies as step 712 described above.

At 814, control circuitry generates a first piece of supplementalcontent using the user information and the supplemental content. In someembodiments, step 814 uses the same or similar methodologies as step 714described above. In some embodiments, the control circuitry generatesthe first piece of supplemental content using a complexity score. Insome embodiments, different complexity scores result in differentsupplemental content being generated. For example, if a complexity scoreis above a first threshold, the control circuitry may generate a firsttype of supplemental content (e.g., in-depth review explaining the firstsegment), and if the complexity score is below the first threshold, thecontrol circuitry may generate a second type of supplemental content(e.g., text box summarizing the scene, laugh track, etc.).

In some embodiments, the control circuitry uses an assisted rewindalgorithm that has the task of determining a user's intention whenperforming a rewind functionality. In some embodiments, the assistedrewind algorithm uses one or more factors to compute or determine acomplexity score of the first segment and replay an augmented version ofthe first segment based on such score. In some embodiments, thecomplexity score is not a numerical value but rather a value thatrepresents a genre (e.g., comedy, sports, etc.), allowing a replayalgorithm to replay the first segment by performing specific actionitems listed in dynamically generated templates for the first segment.In some embodiments, the rewind template is used to augment the contentof the first segment during replay based on suggested actions. Forexample, a replay of a comedy segment with a low complexity score mightresult in automatically augmenting the replay with a background laughtrack (e.g., re-watching the segment feels like watching a sitcom insuch scenario), or displaying various emojis on the screen.

At 816, control circuitry displays the first piece of supplementalcontent to the user. In some embodiments, the supplemental content isoverlaid on the first segment. In some embodiments, the first segment ispaused until the conclusion of the supplemental content. In someembodiments, the first piece of supplemental content is displayedautomatically. In some embodiments, the control circuitry displays alink to the first piece of supplemental content and displays the firstpiece of supplemental content if the user selects the link. In someembodiments, the control circuitry displays the first piece ofsupplemental content using user preferences. For example, the userinformation may indicate that the user prefers supplemental content asan overlay. In some embodiments, the control circuitry displays thefirst piece of supplemental content using any of examples shown and/ordescribed in FIGS. 1A-1C and 3A-3C.

At 818, control circuitry resumes playback of the media asset.

It is contemplated that some suitable steps or suitable descriptions ofFIGS. 6-8 may be used with other suitable embodiments of thisdisclosure. In addition, some suitable steps and descriptions describedin relation to FIGS. 6-8 may be implemented in alternative orders or inparallel to further the purposes of this disclosure. For example, somesuitable steps may be performed in any order or in parallel orsubstantially simultaneously to reduce lag or increase the speed of thesystem or method. Some suitable steps may also be skipped or omittedfrom the process. Furthermore, it should be noted that some suitabledevices or equipment discussed in relation to FIGS. 1-5 could be used toperform one or more of the steps in FIGS. 6-8 .

The processes discussed above are intended to be illustrative and notlimiting. For instance, the steps of the processes discussed herein maybe omitted, modified, combined, and/or rearranged, and any additionalsteps may be performed without departing from the scope of theinvention. More generally, the above disclosure is meant to be exemplaryand not limiting. Only the claims that follow are meant to set bounds asto what the present invention includes. Furthermore, it should be notedthat the features and limitations described in any one embodiment may beapplied to any other embodiment herein, and flowcharts or examplesrelating to one embodiment may be combined with any other embodiment ina suitable manner, done in different orders, or done in parallel. Inaddition, the systems and methods described herein may be performed inreal time. It should also be noted that the systems and/or methodsdescribed above may be applied to, or used in accordance with, othersystems and/or methods.

What is claimed is:
 1. A computer-implemented method for providingsupplemental content, the method comprising using processing circuitryfor: monitoring a plurality of users viewing a media asset for viewinghabits relating to the media asset, wherein the media asset comprises aplurality of segments; determining, using the viewing habits of theplurality of users, that more than a first threshold number of users, ofthe plurality of users, rewound a first segment of the media assetduring playback of the media asset; receiving one or more reviewsrelating to the first segment of the media asset; and in response todetermining that more than the first threshold number of users, of theplurality of users, rewound the first segment and that there are one ormore reviews relating to the first segment, tagging the first segmentwith an identifier indicating that the first segment is a complexsegment, wherein the identifier comprises a number of the plurality ofusers who rewound the first segment.
 2. The method of claim 1, furthercomprising transmitting the media asset comprising the first segmenttagged with an identifier to a user device.
 3. The method of claim 1,wherein the identifier comprises a complexity score calculated using theviewing habits relating to the media asset and the one or more reviewsrelating to the first segment.
 4. The method of claim 1, wherein theidentifier also comprises a number of the plurality of users who pausedthe first segment.
 5. The method of claim 1, wherein the identifiercomprises demographic information related to the plurality of users whorewound the first segment.
 6. The method of claim 5, wherein a majorityof the plurality of users who rewound the first segment have a firstpiece of demographic trait information in common.
 7. The method of claim1, wherein the identifier comprises review information associated withthe one or more reviews relating to the first segment.
 8. The method ofclaim 2, further comprising: receiving, by the user device, the mediaasset comprising the plurality of segments; receiving, by the userdevice, a command from a user to rewind the first segment of theplurality of segments; determining, by the user device, that the firstsegment comprises the identifier; receiving user information related toa profile associated with the user; generating a first piece ofsupplemental content to display to the user based on the first segmentcomprising the identifier, wherein the first piece of supplementalcontent is generated using the user information.
 9. The method of claim8, further comprising: rewinding, by the user device, the first segment;displaying, by the user device, the first piece of supplemental content;and displaying, by the user device, the first segment after the firstpiece of supplemental content.
 10. The method of claim 8, furthercomprising: displaying, by the user device, a selectable option to viewthe first piece of supplemental content; and in response to receiving aselection of the displayed option, displaying, by the user device, thefirst piece of supplemental content.
 11. The method of claim 10, whereinthe selectable option is overlaid on the media asset.
 12. The method ofclaim 10, wherein the selectable option to view the first piece ofsupplemental content is displayed as the first segment is displayed. 13.The method of claim 10, wherein the selectable option to view the firstpiece of supplemental content is displayed as the first segment isrewound.
 14. The method of claim 8, wherein the first piece ofsupplemental content is a first review of the one or more reviewsrelating to the first segment.
 15. The method of claim 8, wherein thefirst piece of supplemental content is a montage comprising a secondsegment of the media asset.
 16. The method of claim 8, wherein the userinformation indicates a first user preference relating to supplementalcontent.
 17. The method of claim 16, wherein the first piece ofsupplemental content is generated according to the first userpreference.
 18. The method of claim 17, wherein the first userpreference indicates that the user prefers supplemental content in avideo format and the first piece of supplemental content is generated ina video format.